‘Bias’, ‘Companion’, ‘Hallucinations’: three examples to understand the significance of the terminology used in and around AI

More generally, language becomes relevant for the way we frame questions. This dialogue is inspired by the notion of “theory as a critical practice of language,” with reference to the work of the German philosopher Günther Anders. Before we can find answers, we first need to agree on questions. Let’s be honest: are we really capable of understanding the reality of technology clearly enough to achieve this? The difficulty is that words sometimes tend to obscure reality rather than help us understand it.

Arnaud Billion: Certain terms commonly used in the field of artificial intelligence influence strategic decisions. They also highlight the errors and weaknesses of the conceptual frameworks within which they are embedded.

For example, it is often said that AI is biased (in terms of its data or its algorithms (2)) to signal that is must be corrected at least its harmful consequences limited. For example, due to the training data, we know that AI-driven CV selection can heavily exclude statistically under-represented groups, such as women or older people. An AI system used by an organisation’s human resources department would therefore be ‘discriminatory by design’.

Björn Fasterling: I agree with Arnaud that, in a technical context, ‘bias’ simply refers to a property assessed against a given benchmark. Similarly, ‘discrimination’ can simply mean distinguishing between different elements. This is something that any automated decision-making system does. It classifies, selects or processes situations differently according to certain criteria.

Arnaud Billion: Yet if every choice is discriminatory, then we would have to resort to drawing CVs at random. This would be a task which AI would perform very well. The point is that the concept of bias is rather pointless, as it encourages us to seek a ‘normal’ answer. However, in many fields, this ‘normality’ is conventional and arbitrary, and not scientifically grounded.

Björn Fasterling: I would like to come back to the terms ‘bias’ and ‘discrimination’. These two terms are commonly used in a normative sense. The moral principle of non-discrimination requires that every person be treated with the same dignity and respect. It prohibits any unjustified differential treatment based on protected characteristics such as skin colour, gender, religion or age… .

In this normative sense, ‘discriminatory bias’ constitutes a deviation from this moral standard. Confusion arises when the different meanings of ‘bias’ are mixed up. A technical bias may be mistaken for an injustice. Unjust discrimination may be reduced to a technical flaw. In that case, everything ends up becoming a ‘bias’, which means we can continue the discussion without really knowing what we’re talking about…

Arnaud Billion: Ultimately, we should ask ourselves: hasn’t this notion of ‘AI bias’ primarily served as an argument for developing and selling new (AI) software designed to combat bias? There are always numerous solutions to a problem that is poorly understood.

Arnaud Billion: How safe it feels to have an AI companion by your side. A highly powerful and knowledgeable entity, like Dr Faust’s Mephisto, who will help me with my administrative tasks, organising my life and my business, or simply support me in my social and emotional life!

Unfortunately, the AI companion, the AI assistant or the AI agent are nothing more than mirages. Even worse: they are mere facades. One need only remember that ‘AI’ is software – or even less than that: a piece of code corresponding to an electronic procedure (3).

‘AI’ is the name we give to a fiction that conceals value chains orchestrated by big tech companies who seek to generate profits from individuals or businesses. The problem with believing in an AI companion is that one thinks that there is an entity within the computer that will obey me and defend my interests. Yet we have no reason to believe this, and we should question the series of misunderstandings that have led us to this belief.

Björn Fasterling: So this “companion” doesn’t really resemble someone with whom we’d share our bread. Perhaps it’s more like a catheter inserted into our brain. Behind this companion, this seemingly privileged access point, lies an entire distributed infrastructure, made up of machines, programs, permissions, and institutional arrangements. Through this catheter, the infrastructure extracts information from our environment and injects content into us to trigger our actions. As if by magic all this complexity disappears behind the illusion of a single assistant, who stands by your side and shares with you the burden of existence.

Arnaud Billion: Beyond that, all these anthropomorphic concepts (learning, hallucination, memory, attention, etc.) are particularly misleading and reflect an irrational and persistent belief, a belief fostered by powerful messaging, or even propaganda, toward which many are very gullible, and which we would do well to question.

Arnaud Billion: When we receive a response from the AI that seems clearly wrong to us, or that we find unsatisfactory, we often refer to it as an “AI hallucination” (4). It’s a bit as if the AI had made a mistake. We think it had been fooled and was providing us with incorrect information. Technically, however, there is no error present at the moment of the so-called “hallucination”. On the contrary, the calculation proceeds as it should.

Björn Fasterling: To hallucinate is to experience something that isn’t there. But there’s no reason to believe that an AI experiences anything at all. It’s, in a way, a squared hallucination. We’re hallucinating an AI that’s hallucinating.

Arnaud Billion: So what now? I'd say we stop hallucinating about a “truth-telling machine”. We have a chance to remember that the output of the software called “AI” will simply be a response calculated based on numerous criteria (5) (potentially reusing my profiling data, my purchase predictions, maximizing my engagement, executing real-time ad targeting, seeking to influence my next vote, etc.)

Above all, a prompt is nothing more than a computer command designed to trigger extremely sophisticated calculations. There is little reason to believe these calculations will be appropriate. In any case, this is a far cry from the everyday computing experience of searching a database configured to provide the expected answer.

Björn Fasterling: You’re talking about the prompt. Now that’s finally an honest term, one that describes almost exactly what’s going on. In a theater, the person we call the “souffleur” in French is known as “the prompter” in English. The prompter whispers the lines to the actor but doesn’t perform in the play. With generative AI, it’s now the human who “prompts.” In other words, the human whispers to the massive computing infrastructure, and it’s the infrastructure that performs on stage!